feat(cli): fast and no more JIT openapi spec - #13
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ac7e6a7566
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@claude review |
There was a problem hiding this comment.
The previously reported blocking issues (missing tsconfig for typecheck, greedy boolean-flag parsing, and the --output null-body bug) all appear fixed in this revision. This round's findings are nits only (duplicated location-mapping/kebab-case logic, and a narrow nested-boolean-flag edge case with an existing --body-json workaround). Beyond those, I checked the codex-reported __schema --json legacy-envelope regression and the duplicated LEGACY_FIELD_FLAGS_UNSUPPORTED set in the test file — both are fine as-is (the README now documents the new discovery contract, and the test's copy is intentional isolation from the runtime source, not accidental drift).
Extended reasoning...
This PR is a large, complex rewrite of the CLI's request pipeline, build process, and CI/release scripts (26 files), including auth-header construction in src/client.ts and a new OpenAPI-contract compiler. Given that scope and the fact that a prior review round surfaced real P1/P2 bugs (now fixed), I don't think this warrants approval outright, but this round's findings are limited to nits with no new blocking issues, so I'm leaving this as a short status note rather than a full deferral write-up.
|
@claude review |
| import { parse } from "yaml"; | ||
|
|
There was a problem hiding this comment.
🟡 The command-naming disambiguator in planCommandNames (conformance/src/naming.ts, wired into the runtime CLI by this PR) produces stuttered/opaque action names for three new 4.10.0 dashboard-placement operations: unstable_dashboards_deletePlacement -> delete-delete-placement, unstable_dashboards_updatePlacement -> update-update-placement, and unstable_dashboards_addPlacement (POST with a path arg) -> the uninformative post. Users must type e.g. langfuse api unstable-dashboards delete-delete-placement <dashboardId> <placementId> — the commands work correctly, this is purely a naming-quality issue.
Extended reasoning...
What the bug is. planCommandNames in conformance/src/naming.ts infers a CLI action verb from HTTP method + operationId suffix, then runs a collision disambiguator when two operations on the same resource map to the same action. For the three new placement operations added by this PR's 4.10.0 snapshot, the generated action names are low quality:
unstable_dashboards_deletePlacement(DELETE /dashboards/{dashboardId}/placements/{placementId}) collides withunstable_dashboards_deleteonunstable-dashboards:delete. The disambiguator strips the resource prefix fromunstable-dashboards-delete-placement, leavingdelete-placement, then re-prepends the inferred action verb (delete) — producing the stuttereddelete-delete-placement.unstable_dashboards_updatePlacementfollows the identical path with PATCH ->update, producingupdate-update-placement.unstable_dashboards_addPlacement(POST /dashboards/{dashboardId}/placements) has a path argument, so thePOST && !hasPathArg -> createheuristic doesn't fire. The operationId suffixdashboards_addPlacementisn't a recognized canonical verb either, soinferActionfalls all the way through tokebabCase(method), yielding the bare HTTP verbpostas the action name. It's unique withinunstable-dashboards, so no disambiguation runs andpoststands as the final name.
Why nothing prevents this today. This logic is new/newly-exercised by this PR (it previously only backed conformance test generation; this PR wires planCommandNames into src/contracts/compiler.ts to drive the actual runtime CLI command surface). The disambiguator's prefix-stripping only checks whether the resource prefix or an action-synonym prefix matches — it never checks whether the already-inferred action verb also appears at the start of the remainder after stripping, so when the operationId suffix itself begins with the action word (deletePlacement, updatePlacement), the verb gets attached twice. Similarly, inferAction's POST-to-create shortcut is conditioned on having no path argument, which is a reasonable general heuristic but leaves any POST-with-path-arg operation whose suffix isn't independently recognized to fall through to the raw HTTP method.
Impact. All three commands are fully functional — I confirmed with the verifiers that the conformance suite drives updatePlacement/deletePlacement successfully via the compiled schema, and each name is unique and routes to the correct operation. The only effect is that users typing langfuse api unstable-dashboards <action> ... see confusing/uninformative action names (delete-delete-placement, update-update-placement, post) in --help output and have to use those exact strings on the command line.
Step-by-step proof (matches independent empirical verification against the compiled 4.10.0 contract by three verifiers):
unstable_dashboards_deletePlacement: resource =unstable-dashboards.inferActionsees DELETE + a path arg present -> action =delete. This collides withunstable_dashboards_delete's action (alsodelete) on the same resource, so the disambiguator runs on the full slugunstable-dashboards-delete-placement. It strips the resource prefixunstable-dashboards-, leavingdelete-placement. It then returns${action}-${name}=delete-delete-placement.unstable_dashboards_updatePlacement: identical shape, PATCH -> actionupdate, collision withunstable_dashboards_update-> final nameupdate-update-placement.unstable_dashboards_addPlacement: POST with path argdashboardIdpresent, so the create-shortcut doesn't apply; suffixdashboards_addPlacementdoesn't match any canonical verb; falls through tokebabCase('POST')=post; unique withinunstable-dashboards, so it's the final action name with no disambiguation needed.
Suggested fix. In the disambiguator, after stripping the resource/synonym prefix, check whether the remainder already starts with the inferred action verb (or a recognized synonym of it) before re-prepending — e.g. only prepend when the remainder doesn't already begin with that token, so delete-placement stays delete-placement instead of becoming delete-delete-placement. For addPlacement, consider extending the verb-recognition list (or the operationId-suffix parser) to map add -> create/add independent of whether a path argument is present, since "add a sub-resource to a parent identified by path" is a common and reasonable pattern that the current no-path-arg restriction doesn't anticipate.
One verifier argued this shouldn't be filed since the outputs are technically valid, unique, and deterministic. I agree the commands are not broken — that's why this is a nit, not a blocking issue — but the double-verb stutter and bare-HTTP-verb fallback are clearly unintended artifacts of the prefix-stripping logic rather than deliberate design choices, and they will confront every user of these three new endpoints, so it's worth a follow-up fix.
| if (parameter.required && target[parameter.name] === undefined) { | ||
| throw new CliError(`Missing required option --${parameter.cliName}`); | ||
| } | ||
| } | ||
| let body = completeBody ?? fieldBody; | ||
| if (completeBody === undefined && operation.requestBody?.legacyFieldFlags) { | ||
| const missing = operation.requestBody.fields | ||
| .filter((field) => field.required && fieldBody?.[field.name] === undefined) | ||
| .map((field) => `--${field.name}`); | ||
| if (missing.length > 0) { | ||
| throw new CliError(`Missing required body option(s): ${missing.join(", ")}`); | ||
| } | ||
| if (body === undefined && operation.requestBody.required) body = {}; | ||
| } | ||
| if (operation.requestBody?.required && body === undefined) { | ||
| throw new CliError(`${operation.operationId} requires a request body`); | ||
| } | ||
| if (body !== undefined) input.body = body; |
There was a problem hiding this comment.
🟡 In parseOperationInput (src/cli.ts), let body = completeBody ?? fieldBody; uses ??, which treats an explicit JSON null from --body-json null/--body-file the same as undefined and silently replaces it with fieldBody. This causes a misleading "requires a request body" error for required-body operations, or silently drops the null for optional ones — fix by using completeBody !== undefined ? completeBody : fieldBody, matching the strict === undefined check the adjacent guard already uses.
Extended reasoning...
parseOperationInput in src/cli.ts builds completeBody from --body-json/--body-file, which can legitimately be the JSON value null (e.g. a user running --body-json null). At the line let body = completeBody ?? fieldBody;, the ?? operator treats null the same as undefined, so an explicit null completeBody gets silently replaced by fieldBody (which is undefined when no legacy field flags were passed).
This produces two distinct, incorrect behaviors depending on whether the operation's request body is required:
- Required body: the subsequent check
operation.requestBody?.required && body === undefinedthrows"<operationId> requires a request body"— even though the user explicitly supplied one via--body-json null. The error message is actively misleading in this case. - Optional body: since
body !== undefinedis false,input.bodyis never set on the call input, so no body is sent at all — silently changing the request from "send an explicit null body" to "send no body," which is a different wire behavior.
Notably, a few lines above this in the same function, the guard that decides whether to fall back to legacy field flags uses a strict completeBody === undefined check — correctly distinguishing null from undefined. This shows the author's intent was to treat an explicit null as a real value, but the very next use of completeBody (the ?? on the body-resolution line) fails to apply that same distinction. It's an internal inconsistency within the same function, not just a stylistic nit.
Step-by-step proof: For an operation with requestBody.required = true, calling parseOperationInput with tokens ["<id>", "--body-json", "null"]:
--body-json nullis parsed viaJSON.parse("null"), socompleteBody = null.- The strict guard
completeBody === undefined && ...evaluates tofalse(correctly, sincenull !== undefined), so the legacy-field-flag branch is skipped as intended. - At
let body = completeBody ?? fieldBody, since??treatsnullas nullish,bodybecomesfieldBody, which isundefined. - The required-body check then throws
"widgets_update requires a request body", contradicting the fact that the user did supply a body. - For the optional-body case,
body !== undefinedis false, soinput.bodyis never set, and the explicit null is dropped entirely.
Why nothing else catches this: only literal JSON null triggers the bug — falsy-but-non-nullish values like false, 0, or "" are not affected by ?? and pass through correctly, so this only surfaces for the specific edge case of an explicit top-level null body.
Fix: replace the ?? with a strict check, matching the pattern already used a few lines above: let body = completeBody !== undefined ? completeBody : fieldBody;
Severity note: all three independent verifiers who examined this confirmed the bug is real and reproducible, but rated it nit because no current Langfuse API operation actually accepts a bare JSON null as its top-level request body — every documented request body is an object (e.g. SubmitFeedbackRequest, CreateScoreRequest). So while the code is genuinely inconsistent with its own adjacent guard and would confuse a user who explicitly tries --body-json null, it doesn't affect any realistic invocation of the CLI today.
No description provided.